Discovering Linear Models of Grid Workload

Despite extensive research focused on enabling QoS for grid users through economic and intelligent resource provisioning, no consensus has emerged on the most promising strategies. On top of intrinsically challenging problems, the complexity and size of data has so far drastically limited the number of comparative experiments. An alternative to experimenting on real, large, and complex data, is to look for well-founded and parsimonious representations. The goal of this paper is to answer a set of preliminary questions, which may help steering the design of those along feasible paths: is it possible to exhibit consistent models of the grid workload? If such models do exist, which classes of models are more appropriate, considering both simplicity and descriptive power? How can we actually discover such models? And finally, how can we assess the quality of these models on a statistically rigorous basis? Our main contributions are twofold. First we found that grid workload models can consistently be discovered from the real data, and that limiting the range of models to piecewise linear time series models is sufficiently powerful. Second, we presents a bootstrapping strategy for building more robust models from the limited samples at hand. This study is based on exhaustive information representative of a significant fraction of e-science computing activity in Europe

Discovering Piecewise Linear Models of Grid Workload

International audienceDespite extensive research focused on enabling QoS for grid users through economic and intelligent resource provisioning, no consensus has emerged on the most promising strategies. On top of intrinsically challenging problems, the complexity and size of data has so far drastically limited the number of comparative experiments. An alternative to experimenting on real, large, and complex data, is to look for well-founded and parsimonious representations. This study is based on exhaustive information about the gLite-monitored jobs from the EGEE grid, representative of a significant fraction of e-science computing activity in Europe. Our main contributions are twofold. First we found that workload models for this grid can consistently be discovered from the real data, and that limiting the range of models to piecewise linear time series models is sufficiently powerful. Second, we present a bootstrapping strategy for building more robust models from the limited samples at hand

Towards Non-Stationary Grid Models

International audienceDespite intense research on grid scheduling, differentiated quality of service remains an open question, and no consensus has emerged on the most promising strategy. The difficulties of experimentation might be one of the root causes of this stalling. An alternative to experimenting on real, large, and complex data is to look for well-founded and parsimonious representations, which may also contribute to the a-priori knowledge required for operational Autonomics. The goal of this paper is thus to explore explanatory and generative models rather than predictive ones: is it possible to exhibit and validate consistent models of the grid workload? Most existing work on modeling grid behavior describes grids as complex systems, but assumes stationarity and concludes to some form of long-range dependence. But the physical (economic and sociologic) processes governing the grid behavior dispel the stationarity hypothesis. This paper considers an appealing different class of models: a sequence of stationary processes separated by breakpoints. The model selection question is now defined as identifying the breakpoints and fitting the processes in each segment. Experimenting with data from the EGEE/EGI grid, we found that a non-stationary model can consistently be identified from empirical data, and that limiting the range of models to piecewise affine (autoregressive) time series is sufficiently powerful. We propose and experiment a validation methodology that empirically addresses the current lack of theoretical results concerning the quality of the estimated model parameters. Finally, we present a bootstrapping strategy for building more robust models from the limited samples at hand